Papers with BERT-based pre-trained language models
LLM-GEm: Large Language Model-Guided Prediction of People’s Empathy Levels towards Newspaper Article (2024.findings-eacl)
Copied to clipboard
| Challenge: | Empathy is a key component of human-to-human interactions, and is often overlooked due to the inherent noise in crowdsourced annotations. |
| Approach: | They propose a large language model-guided empathy prediction system that rectifies annotation errors based on defined annotation selection threshold and makes annotations reliable for conventional empathy prediction models. |
| Outcome: | The proposed system rectifies annotation errors based on defined selection threshold and makes the annotations reliable for conventional empathy prediction models, e.g., BERT-based pre-trained language models. |
ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer (2021.acl-long)
Copied to clipboard
| Challenge: | Existing BERT-based pre-trained language models achieve high performance on many downstream tasks, but native derived sentence representations are collapsed and thus poor performance on semantic textual similarity (STS) tasks. |
| Approach: | They propose a framework for self-supervised Sentence Representation Transfer that adopts contrastive learning to fine-tune BERT in an unsupervised way. |
| Outcome: | The proposed framework improves on the BERT-derived representations by 8% on STS datasets and shows robustness in data scarcity scenarios. |